Prop testing is kind of hard to do, as there is no way of passing correct type info to the the prop test strategy, any thoughts on what could be done?

you can add a new ObjectType to the list for LispTime here:

And then in the strategy function, we can create a strategy that only creates valid time string (or maybe valid stings 90% of the time) and then passes them to the function. That will make the prop testing actually useful.

Another thing I have been considering is creating a proc macro like this

#[defun]
#[elprop(<spec>, <spec>)]
fn time_add<'ob>(a: LispTime, b: LispTime, cx: &'ob Context) -> Object<'ob> {

where you can specify exactly what the valid inputs should be (for example in the base64 case, the function is only meaningful on ascii and raw bytes).

I completely agree about the custom elprop macro, that would allow us to created custom prop testing features. Let me see if I can think of something

I improved elprop again to better handle panics and avoid timeouts. It still does not have special handling for time functions but it works much better now. I was able to find a handful of overflow issues with the time-add function.

I added a new elprop proc macro like we discussed. I pushed an example of using it with time-add to this branch. That should help find any remaining issues in this function. I also added it to base64-encode-string.

Nice, I will continue fixing my functions.

Also was wondering if it would be better to move the elprop tests into tests rather than converting it to json and then using a external runner. I feel that would give more room to create proptests in a easier fashion.

elprop tests into tests rather than converting it to json

not entirely sure what you mean. We can't convert them into Rust tests because we need to feed them to both Emacs and Rune.

I meant why can they not be just normal rust tests, like

mod tests {
  #[test]
  fn elprop_time_add() { .. }
}

We can't convert them into Rust tests because we need to feed them to both Emacs and Rune.

I don't know much about this, so I could be wrong, but I don't understand why that would be problem. I was looking at the Neovim based tests in the Zed editor and they run tests by passing commands to both Zed and a neovim backend in a test I think.

I suppose you are right about that. We could have it just call it from a normal Rust test. However the structure is a little complex. Here is a diagram of all the executables involved in running elprop.

On the top we have a shell script to make sure we are never running on old code. Then we have the top-level elprop binary which parses the Rust source code and extracts all the functions. It is here that we run into an issue; Emacs can't read input from stdin, or send data to stdout (except for batch error messages). But it can connect to stdin/stout of its own subprocesses. So we can't just call Emacs and Rune together to test them. Instead what we do is create a middleware binary called runner.rs. This is called as a subprocess of Emacs and can send data to both Emacs and Rune and compare the results. When it is done, we write out a JSON of the results which gets read by elprop.rs and the program terminates. If you think there is a way to simplify this. let me know!

So we could call elprop from inside a regular Rust test (using std::process::Command), but I don't think we could move all the code into Rune.